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ABSTRACT 


The  stochastic  properties  of  the  output  of  a  multiprogramming 
computer  system  are  studied  by  means  of  a  simple  cyclic  queueing 
model.   It  is  shown  that  output  is  asymptotically  normally 
distributed.   The  parameters  are  determined  by  considering  a 
cumulative  stochastic  process  that  depends  upon  busy  period 
properties;  the  latter  may  be  recursively  determined.   Numer- 
ical examples  are  provided. 


Prepared  by: 


SERVICE  SYSTEM  OUTPUT,  WITH  APPLICATION  TO  MULTIPROGRAMMING 

>v 
Donald  P.  Gaver 

Carnegie-Mellon  University 

1.   Introduction 

It  is  often  mathematically  convenient  and  useful  to  represent 
the  behavior  of  a  computer  system,  or  part  thereof,  as  a  single  server 
queueing  process.   This  is  appropriate  even  when  several  servers  are 
present,  as  in  multiprogramming  situations  involving  cyclic  queues; 
see  Gaver  [  2  ] ,  and  Lewis  and  Shedler  [  3  ] .   Then  the  server  singled 
out  for  particular  attention  usually  possesses  "general"  (non-exponen- 
tial) service  times,  while  the  others  enjoy  simple  Markov-convenient 
properties.   Assuming  this  structure  it  is  often  possible  to  compute 
such  system  characteristics  as  waiting  time  properties  and  server 
idleness  probability,  where  the  latter  depend  upon  server  processing 
rates  and  the  number  of  programs  (customers)  allowed  to  be  present  in 
the  system  simultaneously. 

This  paper  is  devoted  to  studying  the  distribution  of  the  output 
of  the  server  in  such  a  process.   By  this  we  mean  the  following.   Begin- 
ning at  some  moment   t'   the  total  number  of  service  completions,  during 
(t',t'  +  t)   is  observed.   Denote  this  number  of   Z  (t ' , t '  +  t) ,   or  by 
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Z(t)   if  the  process  has  stationary  increments.   We  term  Z   the  output 
of  our  server,  and  seek  to  characterize  its  behavior.   It  will  be  shown 
in  the  sequel  that  in  interesting  cases   Z   is  approximately  normally 
distributed  as   t   becomes  large.   Furthermore,  in  cyclic  models,  e.g., 
for  multiprogramming,  a  simple  continuity  argument  shows  that  the  out- 
puts of  both  servers  enjoy  the  same  limiting  normal  distribution.   The 
methods  employed  make  possible  a  comparison  of  various  multiprogramming 
situations.   Some  limited  numerical  illustrations  are  presented. 


2 .   Inputs  and  Outputs 

In  this  section  we  record  a  simple  observation  upon  which  much 
of  the  later  development  rests. 

(a)   The  M/G/l  Service  System.   Here   A   denotes  the  Poisson  arrival 
rate,  and   S   is  a  generic  service  time.   Assume   E[S2]  <  °°.   Let 
N(t)   denote  the  number  of  customers  in  the  system  at  time   t,   and 
let  the  input ,   A(t) ,   be  the  total  number  of  arrivals  to  have  occurred 
in   (0,t).   Then  Z(t),   the  total  output  in   (0,t),   is  defined  by 
the  continuity  relation 


N(0)  +  A(t)  =  Z(t)  +  N(t) 


(2.1) 


Now  suppose   p  =  AE[S]  <  1.   Then,  for  large   t,  N(t)   is  finite  while 
A(t)   becomes  large,  and  hence   Z   is  asymptotically  similar  to  A(t). 
Putting  this  formally,  write  (2.1)  as 


A(t)  -  At 

/At? 


Z(t)  -  At 

/aT 


N(t)  -  N(Q) 

/aT 


(2.2) 


Now  when  p  <  1   and   E[S2]  <  °°  it  is  well-known  that 


lim  E[N(t)|N(0)]  =  E[N(»)]  <  °° 
and  by  Tchebychev's  inequality, 


(2.3) 


p{  |N(t)  -  N(0)|  >  £}  <  E[N(t)1+  N(Q)  ^  Q 
/At  e/AT 


(2.4) 


for  any   e  >  0  as   t  ->  °°.   Thus  it  follows  that  the  left-hand  side 

of  (2.2)  approaches  zero  in  probability,  and  hence  since  the  distribution 


of    UJ   — t   converges  to  the  N(o,l)   law,  so  does  that  of 


7Xt  /u 

This  result  will  hold  true  for  many  types  of  queueing  systems,  e.g., 

for  the   GI/G/1   as  well  as  for  various  multiple  server  configurations. 

(b)   The  Cyclic  System.   Of  particular  interest  in  the  multiprogramming 

computer  system  studies  context  is  the  cyclic  arrangement  depicted  in 

Fig.    1. 


-*<ac 


H  CPU  | >■  ?*pco  t>  I  dtu 


Figure   1 

In  the  most  rudimentary  model  a  fixed  finite  number,   J,   of  tasks  or 
programs  is  present  in  the  system  at  any  time.   A  program  is  processed 
at  the  Central  Processing  Unit   (CPU)   until  an  interruption  occurs 
("page  interrupt"  in  certain  types  of  machines)  for  lack  of  information. 
At  this  moment  the  program  enters  the  Data  Transfer  Unit   (DTU)   stage, 
where  it  awaits  and  eventually  receives  the  required  information  and 
is  then  returned  to  the   CPU   stage.   In  the  mean  time  the   CPU  may  be 
busy  processing  another  program,  and  is  therefore  kept  busy.   Programs 
that  are  completed  at  the   CPU   stage  are  assumed  to  leave  the  system 
and  be  instantaneously  replaced.   Models  of  this  type  have  been  consid- 
ered by  various  authors,  cf . ,  Gaver  [2],  Lewis  and  Shedler  [3],  and 
Shedler  [  4  ] . 

Although  only  a  limited  amount  of  actual  data  analysis  has  been 
carried  out,  it  is  apparently  roughly  appropriate  to  assume  that  the 
service  times  of  programs  at  the  CPU  are  independently  and  exponentially 
distributed.   Service  times  at  the  DTU  are  of  non-exponential  (more 
nearly  constant)  character. 


We  shall,  as  a  consequence,  make  such  assumptions;  it  is  noted  that 
this  represents  a  reversal  of  the  assumptions  made  in  Gaver  [2], 

In  order  to  discuss  the  outputs  of  the  servers,  denote  by  C(t) 
the  number  of  programs  that  complete  the  CPU   stage  (number  of  CPU 
service  completions)  in   (0,t),  while  D(t)   refers  to  the  corre- 
sponding quantity  for  the  DTU.   In  the  present  model  the  programs 
remain  the  same  and  there  are  no  actual  departures  from  the  system. 
Later,  we  put  in  a  departure  mechanism.   Next,   N  (t)   and  N  (t) 
represent  the  number  of  programs  at  the  CPU  and  DTU  stages  re- 
spectively.  Then  again  the  continuity  relation  (where  C   takes  the 
place  of  Z)   states  that 

Nc(0)  +  D(t)  =  C(t)  +  Nc(t)  (2.5) 

Hence  if  there  exist  norming  constants   y  and  a  such  that 
[D(t)  -  yt](a/t)    has  a  limiting  normal  distribution,  then,  by  the 
same  argument  as  that  outlined  in  connection  with  the  M/G/1-system, 
[C(t)  -  Mt](a/t)    approaches  the  same  normal  distribution.   That 
is,  the  output  distributions  of  CPU  and  DTU  are  asymptotically 
identical.   But  for  the  present  model  it  is  evident  that  D(t)   is 
actually  a  cumulative  process  in  the  sense  of  Smith,  cf.  Cox  [1],  or 
Smith  [5].   Hence  asymptotic  normality  follows,  e.g.,  from  Smith's 
development  ([5],  pp.  262-263]). 
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3.   The  Cyclic  System,  Its  Busy  Periods,  and  a  Cumulative  Process. 

The  process  associated  with  the  cyclic  system  of  Figure  1  may 
conveniently  be  viewed  as  a  succession  of  busy  and  idle  periods  for 
the  DTU.  Let  there  be  J  programs  circulating,  and  consider  a 
moment  t'  such  that  N  (t'-0)  =  0  but  N  (t1)  =  1.  That  is,  the 
system  is  idle  prior  to  t',  becoming  busy  with  the  service  of  one 
customer  at   t'.   A  busy  period  for  the  DTU   is  defined  by 

T;L(J)  =  inf{t  :>  0|ND(t+t')  =  0}  (3.1) 

The  subscript  indicates  that  one  program  is  present  at  the  start  of 
the  busy  period.   Following  each  busy  period  is  an  idle  period,  during 
which  all  J  programs  are  queued  behind  the  CPU.   By  the  Markov  prop- 
erty of  the  CPU  service  times,  a  generic  idle  period  of  duration  I 
is  exponentially  distributed  with  mean  A   .   Successive  idle  periods 
and  subsequent  busy  periods  are  independently  and  identically  distri- 
buted random  variables.   Put 

x(n)  m   r(n)  +  T(n)(J)   for  n=lj2>...  ;  (3.2) 

{X   }   represents  the  times  between  the  successive  regeneration 
points  at  which  the  DTU  becomes  idle.   In  terms  of  the   {X    , 
n  =  1,2,...}   sequence,  which  is  one  of  independently  and  identically 
distributed  random  variables,  one  can  speak  of  a  renewal  counting 
process,   R(t) ,   where 

R(t)  =  0   iff   X(1)  >  t 


(3.3) 


T 

n=l  n=l 


R(t)  =  j   iff    I     X(n)  s  t   and    \     X(n)  >  t 


for  j  =  2,3,...  . 


Let  the  output,  or  number  of  service  completions,  during  a  (the 
n — )  busy  period  be 

e{n)(J)  =  D(t^n)(J))    for  n-  1,2,3 (3.4) 

Then  the  output  process  D(t)   is  cumulative  in  the  sense  of  Smith  [5], 
with 

R(t)  M 
D(t)  «  I      e|n;(J)  (3.5) 

n=l 

as   t  ■>  oo   (the  approximation  consists  in  neglecting  outputs  during 

part  of  an  X-cycle;  these  are  negligible  for  large   t) .   A 

central  limit  theorem  for  such  processes,  cf.   Cox  [  1],  enables  one 

to  show  that  D(t),   appropriately  normalized,  is  approximately  normally 

distributed  for  large   t,   and  to  find  the  parameters  of  the  limiting 

distribution  (asymptotic  mean  and  variance)  explicitly  in  terms  of  the 

CPU  service  rate,   A,   and  the  distribution  of  service  times  at  the 

DTU.   To  be  specific,  it  may  be  shown  that  as   t  -*•   °° 

E[6,(J)] 
E[D(t)]  ~  t    E|x]    e  t  u  (3.6) 


and 


Var[D(t)]  ~  t\ zrrvi + 


Var[61(J)]        Var[X](E[31U)])2 


E[X]  (E[X]) 


CovfB,  (J),X]E[B1  (J)] 

"2 lilxTP 
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=   o2   t 


and  that  — ^— tends  to  the        N(0,1)   law  as   t  becomes 

a/t 
large.   Explicit  evaluation  of  the  parameters   (j   and   a2   is  discussed 


in  the  next  section.   Finally  the  development  of  Section  2  then  implies 

that    ^  "  ty   also  has  the  limiting  N(0,1)   distribution. 

o/t 


4 .   Cumulative  Process  Parameters 

Recursive  evaluation  of  busy  period  properties  as   J   increases 
was  discussed  in  Gaver  [2].   Let   S  be  the  DTU  service  time,  with 
distribution  U(x)and  Laplace-Stielt jes  transform 


u(s)  = 


e  SX  U  {dx}. 


0- 


Then  consider  the  following  cases. 

Representation. 

(A)     J  =  1.   Here  clearly 


(4.1) 


t1(D  =  S 


B1(D  =  1. 


(4.2) 


(B) 


J  =  2.   Condition  on  S   to  find  that 

Probability 
-XS 


1  -  e 


-XS 


tx(2) 


61(2) 


s(=x1(D) 


S  +  t]_(2) 


1  +  Bj_(2) 


Figure  2. 
Use  of  the  symbol   '   means  that,  for  example,   t' (2)   has  the  t,  (2) 

distribution  but  is  independent  of  events  leading  up  to  the  initial 

service  completion.   To  explain  further,  consider  the  situation  just 

following  the  initial  service  completion  of  the  busy  period.   Either 

-XS 
(i)   no  CPU  output  occurred  during   S,   an  event  of  probability  e    , 

in  which  case  the  busy  period  is  of  duration   S  with  one  output,  or 


-XS 


(ii)   exactly  one  CPU  output  occurred,  an  event  of  probability  1  -  e    , 
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in  -  tich  case  the  initial  situation  '^s  reproduced,  with  one  program 
at  "he  CPU  and  one  at  the  DTU ,  out  wi>h  an  initial  c<  iponent  of  busy 
period  duration,   S,   and  one  inif'^1  output. 

(C)     Arbitrary  J.   Again  :onc ition  on   S.  Define   x  .  (J)   ar.d   3.  (J), 
(i  =  1,2,..., J)   to  be  respectively  the  first  passage  time  from   i   to 
i  -  1   and  the  output  therein,  given  that  a  service  is  just  commencing 
ex.     the  moment   N   =  i. 

For  the  present  setup  we  < hen  have 

Probe   lity 
-AS 


ASe 


(AS)Je 


1  -  e 


J-2     i 
■XS  v   (AS)J 


x1(J) 


B1(J) 


■i   1 

J 

— . 

j  =U 

J-.1 

s 

S  +   t|M) 

i= ! 

'  i  ) 

S  +      ;      t'.(J) 
i=l      " 

]_ 

1  +   3[(J) 

J 
1   +      } 
i=l 

i 

J-l 

i  +   I    e:u) 
i=i    x 

Figure  3. 
Now  a  little  reflect  *n~    shows  that   t.(J)   has  the  same  distribution 

as   t  (j-i+1)  ,   and  similarly  that   3.  (J)   has  the  sane  distribution  a 

3-.  (J-i+1)  .   This  fret   enables  us  to  successively  compute  the  various 

expectations  required  to  evaluate  (3.6)  and  (3.7).  V'i   now  illustrate. 

Expectations . 

(A')     J  =  1.   Directly, 


E[x1(l)]    =  E[S],  Var[r1(l    1    =  Var[S] 


E[B1(D]    =   1 


Var[B-(l)]    =   0 


(4.3) 


Cov[T1(l),31(Dn    =   0 
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(B1)    J  =  2.   Conditional  on   S, 

E[t1(2)|S]  =  S  +  (1-e  XS)E[t1(2)].  (4.4) 

Consequently  after  removal  of  the  condition  on   S   and  use  of  (4.1). 


E[Tl(2)]  =  -  -^J--  HU  (4-5) 

Likewise, 


E[e-AS]  '  u^ 


E[B-(2)]  =  \=-   =  -iy  (4.6) 

Squaring  column  by  column  in  Fig.  2  delivers  second  moments.   For  example, 


(4.7) 


E[tJ(2)|S]  =  S2e  AS  +  E[(S  +  t|(2))2](1  -  e  XS) 

=  S2  +  {2S  E[Tl(2)]  +  E[t2(2)]}(1  -  e~AS). 


So,  upon  removal  of  the  condition  on  S, 


E[S2]  4-  2E[S(1  -  e~XS)]E[T1(2)] 

E[t2(2)]  - ± ;       (4.8) 

1  E[e_XS] 


the  value  of  (4.5)  is  introduced  to  evaluate  the  latter  expression. 
Next  the  variance  is  computed  by  subtracting  off  the  square  of  (4.5). 
In  analogous  fashion 

1  +  2  E[(l  -  e"XS)]E[6  (2)] 

E[B2(2)]  = i ;  (4.9) 

1  E[e"XS] 

insertion  of  (4.6)  and  subtraction  of  its  iauare  yields  the  variance. 
The  covariance  is  obtained  from  the  expectation 
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E[S]  +  E[S(l-e  XS)]E[g  (2)]  +  E[l-e  XS]E[x1(2)] 

£[6,(2)1  (2)]  =  ~-r ± (4.10) 

1    1  E[e"XS] 


by  subtraction  of  the  product  of  (A. 5)  and  (4.6).   These  expressions 
can  be  evaluated  in  terms  of  the  transform  (4.1)  and  its  derivatives, 
and  thus  there  is  natural  impetus  to  employ  some  explicitly  transformable 
density,  e.g.,  the  gamma  or  hyperexponential ,  to  represent   DTU   service 
times . 

Examination  of  Fig.  3  makes  it  clear  that  the  busy  period  moments 
for  any   J   can  be  expressed  in  terms  of  the  corresponding  moments  for 
smaller  J-values.   This  step  can  perhaps  be  best  carried  out  numerically, 
for  neat  closed-form  exact  expressions  will  not  occur. 

The  busy  period  moments  obtained  by  the  procedure  described  may 
be  employed  to  evaluate  the  output  parameters  (3.6)  and  (3.7).   Some 
numerical  illustrations  are  given  in  the  following  section. 
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5 .   Numerical  Examples 

The  effect  of  assuming  various  parameter  values  in  our  multi- 
programming model  can  be  investigated  numerically  by  putting  the  results 
of  the  previous  section  to  work.   Some  rather  limited  examples  appear 
in  the  following  table. 


CPU 

A: 

0.20 

0.5 

1.0 

1.5 

2.0 

5.0 

DTU 

P 

0.17 

0.33 

0.50 

0.60 

0.67 

0.83 

Constant,  E[S]=1; 

J-i; 

a2 

0.12 

0.15 

0.13 

0.10 

0.07 

0.02 

M 

0.20 

0.45 

0.73 

0.87 

0.94 

0.98 

J=2; 

a2 

0.18 

0.31 

0.23 

0.12 

0.05 

0.01 

u 

0.17 

0.33 

0.50 

0.60 

0.67 

0.83 

Exponential,  E[S]=1; 

J=i; 

a2 

0.17 

0.19 

0.26 

0.33 

0.35 

0.52 

y 

0.19 

0.43 

0.67 

0.79 

0.86 

0.92 

J=2; 

a2 

0.17 

0.29 

0.37 

0.47 

0.56 

0.81 

Figure  4. 
Notice  that  when   A  >  (e[S])    =  1,   in  which  case  the  DTU   stage 
acts  as  bottleneck,  the  move  from  J  =  1   to  J  =  2  has  dramatic  effects, 
Although  the  output  rate  can  never  exceed  unity,  improvements  of  at  least 
ten  percent  occur.   The  addition  of  further  programs   (J  >  2)   is 
apparently  justified  only  if  considerable  overhead  activity  is  present; 
this  feature  is  not  included  in  the  present  model.   It  is  of  interest 
to  compare  the  numerical  values  of  Figure  4  to  those  obtained  by  Shedler 
[4].   Clearly   A [CPU  utilization]  =  u,   and  a  reference  to  the 
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appropriate  entries  in  Table  1  of  [  4  ]  provides  numerical  confirmation. 

Examination  of  the  variance  of  output  is  of  some  interest.   If 
X      is  relatively  small  (CPU  the  bottleneck)  it  appears  that 

(a)  for  J  =  1,   Var[C(t)|S  exponential]  >  Var[c(t)|S  constant] 
but 

(b)  for   J  =  2,   Var[C(t)|s  exponential]  <  Var[C(t)|s  constant] 
By  way  of  explanation,  one  sees  that  when  J  =  2   busy  periods  are  more 
likely  to  involve  more  than  one   DTU   service  when   S   is  constant  than 
when   S   is  exponential.   Of  course,  if  X   >  (E[S])     the  DTU  becomes 
the  bottleneck.   As  anticipated  in  this  situation  the  output  behaves 

like  a  renewal  process  with  inter-event  times  distributed  according  to 

?     -1 
S.   Consequently  when   S   is  constant,   a   =  t   Var[C(t)]   dwindles  to 

zero  as  X      increases,  reflecting  the  fact  that  outputs  through  the 

DTU  bottleneck  are  regular.   Of  course,  the  regularity  is  even  greater 

when  J  =  2   than  when   J  =  1.   If ,  on  the  other  hand,   S   is  exponential 

the  variance  gradually  approaches  that  of  the  DTU  bottleneck,  namely 

unity. 

It  may  be  guessed  from  the  numbers  of  the  last  table  that  when 

X   >    1   the  assumption  of  exponential   S   provides  an  underestimate  of 

output  rate   u,   and  an  overestimate  of  a2,   provided   S   is  more 

regular — of  smaller  variance — than  the  exponential.   Another  estimate 

of   a2,   useful  when  the  DTU  rate  is  smaller  than  that  of  the  CPU, 

is  obtained  by  simply  assuming  that  the  DTU   is  never  idle,  and  thus 


2      Var[S] 

~  TeTsTF 
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a  familiar  renewal  theory  result  that  will  become  increasingly  accurate 
for  larger  and  larger  J.   Although  we  do  not  explore  such  approxima- 
tions further  at  this  point  it  seems  evident  that  for  increasingly 
complex  systems — those  in  which  there  are  considerations  of  overhead, 
non-exponential  distributions,  and  in  which  J  >  2 — the  only  practical 
route  to  understanding  is  through  approximations  and  bounds.   If 
simulations  are  undertaken  it  is  useful  to  have  some  idea  of  the  variance 
of   C(t)   so  that  run  lengths  may  be  established.   Approximate  variances 
are  often  adequate  for  such  purposes. 
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6 .   Program  Termination  and  Output 

The  previous  development  takes  no  account  of  the  fact  that 
individual  programs  actually  terminate.   In  order  to  introduce  this 
effect  into  the  model,  we  can  assume  that  each  time  a  program  leaves 
the   CPU   stage  one  of  two  events  occurs:   (i)  the  program  terminates 
or  is  completed,  or   (ii)  the  program  experiences  an  honest  page  fault 
and  must  go  to  the  DTU   stage.   Suppose  that  choice  of  event  (i)  or 
(ii)  is  governed  by  a  Bernoulli  trials  process  so  that  with  probability 
p   the  program  terminates,  and  with  probability   q  =  1  -  p   the  program 
continues  to  the  DTU   stage.   In  order  to  allow  use  of  the  previous 
analysis  we  shall  assume  that  in  case  event  (i)  occurs  a  new  program 
is  immediately  introduced  into  the  system  at  the  DTU   stage;  the  first 
pass  through  this  stage  may  well  represent   I/O   activity  on  behalf  of 
this  newest  program. 

Let  M(t)   represent  individual  program  output  over  time   t.   Now 
given  C(t),   M(t)   is  conditionally  binomial,  with 


so 


E[M(t)|C(t)]   =   PC(t)  (6.1) 


E[M(t)]  =  PE[C(t)]  -put  =  uMt  (6.2) 


as      t  -*■  °°.      Furthermore, 

Var[M(t)]    =   pqE[C(t)]    +  p2Var[c(t)] 


(6.3) 


~  pqut  +  p2a2t  =  a2t 


and  since  M(t)   is  easily  seen  to  be  a  cumulative  process  the  previously 
quoted  theorem  shows  that  the  actual  output  of  completed  programs  is 
approximately  normal  as   t  -+■  °°. 
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